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Tlrifr-ie^-a-pe poft-e f ingoing MIT research in telerobotics (vehicles capable of come autonomous sensing and 
manipulating, having some remote supervisory control by people) and teleoperation (vehicles for sensing and 
manipulating which are fully controlled remotely by people). Our jaboratory^as^engaged-in- research on both 
humanand automatic control of such^mahipUlators and vehimbs^or over .twenty years. / 
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The current efTort3 JUj ^w-h:ieh*»are-4d«fttifie4»-‘feelQW- by-^the -individual graduate” students doing -the -work-, mix 
human and artificial intelhgence/control: Th a, l uj^r o po e bdbcusa cs-khe idea of adjustable impedance at either 


end of pure niaster-slave teleoperation. The- 
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1. OPERATOR\ADJUSTABLE IMPEDANCE IN BILATERAL REMOTE MANIPULATION 
G. Jagannath Raju k 

When humans manipulate objects in their environment, two senses that are extensively used arc vision 
and the "muscle senses^ that mediate kinesthesia and proprioception. Sometimes skin senses that mediate 
pressure and temperature, are useful. The assumption made here is that the objective c>r the manipulation 
task is to identify. and/or\alter the location of an object in the environment. The ultimate goal in remote 
manipulation would be theVomplete transparency of the interface, i.e., the actuators which transmit the 
command and the sensors that return the required information. The term "telepresence" reflects this concept 
of transporting the human operator not in body but in sensation to the remote location. Though the "skin 
senses" may be blocked by a telconanipulator mechanism and/or the telecommunication channel, the "muscle 
senses" and vision may be Vplaccd with high fidelity transmission channels of vision and 
force/displacement. In reality owiW to limitations imposed by the environment (distances, transmission 
medium) and technology (sensor resolution, transmission bandwidth, time delays) the transmission signals 
are degraded and have to be enliancetJ or compensated for in some way to be of real value. A force feedback 
channel can provide the operator with values of the force levels in the interaction with the object, 
displayed on a screen, or better still Vonvert these measurements back to a force level through a servo- 
actuator to redirect a sense of "feel" to tint operator. The force transmission channel in the forward direction 
transmits the forces that the human operator would have imposed on the task had she been able to manipu- 
late the object directly. The primary objective of this research effort is to study the effect of 
allowing the human operator the option u> select the dynamics of the force transmission chan- 
nel to suit personal muscular characteristics and the task being performed in the remote loca- 
tion. The dynamics of the channel is characterized by a set of impedances. Force(K) and volocity(V) vari- 
ables in mechanical systems are analogous to volihge and current signals in electrical networks, hence 
mechanical impedance and admittance can be defined sihiilarly. 




of teleoperator/telerobotic systems which have more than six degrees of freedom (e.g., a combined vehicle and 
arm, each with five or six D O F) V T he_tJur.d.-r.eport-briuily -d escribes > new cable-controlled parallel link arm 
which offers many advantages over conventional arms for space. x T4w-f^i4h-stmwr»«««>»-ovH fc wopk.oveF-r-eeeiri 
years'On predictor displays to compensate for time delay in tclcdperator loops. 

use of state estimation to help human control decisions in space, \T4io nixth rrffinrtr de&ewbee^otigoi ng research 
in supervisory command language,. Fi nally , repeal, iv coniposit^actlvlLy^oWt^r'-grffcnrifte'stu- 
deata, d ese pt bea * efforts to build a human "fiyable" real-time dynahiic computer-graphic telerobot simulator/! 1 '- 
These projects represent most, but pot all, of the telerobotics research in our laboratory, supported by JPL, 
NASA Ames and NOAA. 




Nomenclature 

J, B . K 
T, /?, 9 
Z, Y : 


Inertia, Damping and Stiffness parameters 
Torque, Velocity and Position 
Impedance and admittance 
Complex frequency 


Subscripts m.a,h,t refer to master, slave, human and task respectively. 
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For a tumped parameter model of a simple mass, dashpot, spring system, the impedance transfer function is : 

z <‘>" no - A+fl+ f 

and the admittance transfer function is : 

^ “ Ji*+Bs+K 

Model of Master-Slave System 

The sub-systems that comprise a single degree of freedom MSM system are depicted as : 



A human operator manipulates a master arm/joystick to specify the desired trajectory of a stave arm in the 
remote location. In addition to driving the actuator of the slave arm, an actuator is used on the master arm 
to reflect the torque that the slave arm exerts on the environment back to the human operator. This is the 
force-feedback signal that the operator uses for adaptation. The three sub-systems are : 

i. The human operator, who translates a given task into a trajectory of her arm, computes the forces 
required to achieve this trajectory, exerts these forces on the master arm, and adjusts her arm 
impedance and the impedances of the MSM, based on force- feedback and/or visual feedback from the 
remote location ; 


ii. A MSM (composed of master arm and slave arm links, and master and slave servo-mechanisms) which 
transmits forces and motion between the human operator and the remote environment, and ; 

iii. The task object in the remote environment that is being manipulated by the slave arm. 


Using a linearized model the dynamics of the operator's arm can be represented by : 

Jkfik+BU)nk+KU)e k -PU)-T„ t - r*(/) 

where / is the neural firing rate ; and the admittance as : 

} , „ _L = Art*) f 

A ” Z A = T*(/K«) “ J h s 2 +B(f)s+KU) 

The operators arm when viewed from the master end of the MSM appears as an impedance Z A and an effort 
(equivalent to voltage in electrical circuits) source P(f ). 

A generalized mass-dashpot-spring model is used to represent the object being manipulated by the slave arm. 
Then its admittance can be represented as : 

y- = = i? 

Zj J t s~+B t s+K t 

When viewed from the slave end of the MSM, the task looks like a passive impedance Z t . 
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The MSM may be modelled as a two-port mechanical system, analogous to two-port electrical networks. At 
the input or master port the MSM interacts with the operator, at the output or slave port with the task 
object in the remote environment. 


The elements of the impedance matrix (* l; ) that characterizes the two port element can be represented as : 
j*lt 

inrhi * 22 ] W 

where 



It can be shown that on combining the dynamics and the feedback (gains represented by fyj) control : 
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Therefore by proper choice of the feedback gains £,y, the elements z,y can be independently modulated to 
match desired specifications. The impedances of the two ports of the MSM in terms of the elements z tJ - can 
be expressed as : 


7 T m {s) ~ * 12*31 

" “ /?„(•) ” ‘22+Z, 

„ T t {s) * 12*21 

* n a {s) 22 z n +z K 


Then given desired master and slave port impedances, Z m and Z t , *n and z 22 can be computed by solving 
the two nonlinear algebraic equations above. For a given task the human operator can adjust the port 
impedances by specifying desired values, and appropriate feedback gains are computed to implement the con- 
trol algorithm. 


One of the proposed experimental tasks emulates the action of a toggle switch with three or more '’table posi- 
tions. If a human were to directly operate the switch, without visual feedback, she would impose a force on 
the switch lever and keep increasing it till the reaction force from the switch mechanism is overcome and the 
switch moves to the adjacent stable position. The human senses that the switch is in the process of moving to 
the next position when the reaction force from the switch mechanism relaxes. The performance of the human 
subject in being able to shift the switch at the slave end of a one degree of freedom MSM using the master 
arm will be studied for different values of both master and slave port impedances. This experiment may illus- 
trate the feasibility of allowing the human operator to adjust the MSM impedances as needed for different 
tasks or sub-tasks. 


2. INVERSE KINEMATICS FOR REDUNDANT SYSTEMS 

Hari Das 

Objective 

Kinematically redundant systems 't.e. systems with more degrees of freedom than required for end 
effector specification, are studied in this project. The purpose is to obtain position of each degree of freedom 
for a given end effector position (a.k.a. the inverse kinematic problem). An example of a redundant system is 
a three link manipulator in two dimensional space where only the two coordinate position of the end point is 
to be specified. The system has one remaining degree of freedom after fixing the end point position. 

Method 

In this method, the inverse kinematics is solved by using the extra degrees of freedom to satisfy physi- 
cal constraints such as obtaining desired manipulator configurations or avoiding obstacles. The method is to 
simulate a dynamically simple but kinematically equivalent system. Since the kinematics involves only 
geometry, any convient dynamics may be chosen for the simulation. Forces are made to act on this simple 
system to drive the end point to the desired end position. Constraints such as a prefered manipula- 
tor shape is obtained by applying torques on each joint proportional to the difference between the 
actual and desired joint angle. Obstacle avoidance is achieved by having repulsive forces act between 
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obstacles and each joint of the manipulator. Advantages of this method are that a human user can 
Interactively choose solutions in a physically intuitive way. With this method, the manipulator end 
effector could be kept within the viewing range of a video camera or could be kept within the range of its 
Joint motion. 

Work Done 

A number of variations of the algorithm using first and second order dynamics have been attempted 
on a three link manipulator in two dimensional space. Also studied were techniques to speed up com- 
putation and insure stability. A number of control algorithms for computation of force at the manipulator 
end point and obstacle- manipulator repulsion were investigated. Convergence to desirable solutions, 
numerical stability of the simulation and computation time were the criteria used to compare the 
different variations. The most promising of these is a first order simulation using a control law on the mani- 
pulator that has a force from the tip of the manipulator to the desired end position and has all other con- 
straints on the motion of the manipulator projected onto the nullspace of its jacobian thereby not 
affecting the end point motion. Trial runs with this variation on a ten link manipulator 'in two dimen- 
sional space have been done. The figure below shows the graphical output of the program with the ten 
link manipulator at the start and end of a simulation where a desired end point trajectory through the 
obstacles and obstacle positions were fed to the algorithm. ( This might be done by a human operator 
in the context of controlling a manipulator arm.) 



Further Work 

The difference equations in the digital implementation of the simulation must be studied for a 
complete numerical stability analysis. The method is to be applied to other redundant systems, for 
example, a vehicle-manipulator system. As a part of this research, a comparison between this method 
and other more conventional techniques is planned. 


3. CABLE-CONTROLLED PARALLEL-LINK MANIPULATOR 
Samuel E. Landsberger 

A new kinematic design for a six DOF parallel-link manipulator arm has been designed and built and 
has undergone preliminary tests. It demonstrates the following advantages over conventional serial-link arms: 
(1) higher stiffness-t >weight ratios, because no bending members are required; (2) higher end-point force 
capability because links act in parallel; (3) ail actuators are mounted on the base, thereby reducing inertia by 
a large factor; (4) inertial properties are invariant with respect to position; (5) cable capstan drive eliminates 
gear and joint backlash; (8) all six cable and actuator subsystems are identical, making manufacture cheaper; 
(7) cable links and spine collapse to a small rest-size; (8) the compressive spine can be preset to any desired 
compliance, or its compliance can be adjusted within the task execution cycle, (<l) the forward loop 
specification of actuator position, given desired end-point position, is an easy calculation and does not require 
inverting a jacobian. 

Disadvantages relative to a serial link manipulator are: (1) the arm cannot "bend around" objects; (2) the 
arm's strength deteriorates at greater than 45 degrees From the symmetry (vertical spine in the figure) posi- 
tion; (3) "wrist” rotation is limited to about fifty degrees. 

Computer synthesis of path (continuous determination of cable length) is presently achieved by dividing the 
desired path into sufficiently small, equally spaced intervals in radius-longitude-inclination space to keep the 
path smooth and within bounds. Present computational steps are approximately 0.1 inches part, yielding a 
speed of about 100 inches per second. 
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CabLe controlled parallel link manipulator 


Analog or digital filtering to smooth out commands as well as various mechanical changes are being con- 
sidered. Further performance testing with regard to bandwidth, acceleration and load characteristics, accu- 
racy and slow speed characteristics is being done. 


4. PREDICTOR DISPLAY COMPENSATION FOR TIME DELAY 
Forrest Buzan 

It has long been appreciated that time delay in a control loop produces instability. Experiments by 
W'.R. Ferrell in our laboratory in 10(55 resulted in a model predicting the number of discrete "open loop 
move, wait for feedback" cycles as a function of time delay and accuracy requirements of the task com- 
ponents. He showed that it is essentially not possible to have delayed force feedback to the same hand as is 
controlling a master-slave teleoperator and avoid instability. 

Recent developments in fast "frame grabber" video technology have enabled me development of a predictor 
display wherein the human operator’s control signals drive a kinematic model of the manipulator: this super- 
poses on the delayed video display a "stick-iigure model" which, because it is instantaneous, leads’ the video 
image of the manipulator arm. This is illustrated in the figure below. 

A series of experiments has demonstrated the efficacy of the predictor display. First experiments by M. 
Noyes used the predictor in two tasks, a tracking task and a task involving picking up blocks and dropping 
them into a box. There were nine treatments: 1 - I used a predictor display while 5-8 did not. Treatments 
l - 8 had a 1.8 second time delay where in l. ? 7, 8 it was continuously refreshed; in 3, 4, 5, 5 it was buffered 
to present same-time frame snapshots. Resolution was high in I, 3, 5, 7; it was low in 2, 4, 6, 8. Treatment 9 
had no predictor, no delay and high resolution. Noyes’ results shown below (means of four subjects) indicate 
a major advantage with the predictor. 
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5. AIDING HUMAN OPERATORS WITH STATE ESTIMATES 
James B. Roseborough 

There are many situations where it would be desirable to have computer based estimates of process 
states to augment an operator's own personal estimates when making decisions. Possible applications of this 
are in space, where the process is a tumbling satellite which the operator must retrieve, and undersea naviga- 
tion where the operator must navigate successfully in an unfamiliar environment. The computer based aid 
may be used to "track" the satellite or the environment, predict future states of the process to help in plan- 
ning, and possibly present alternate views of the process to the operator which would not normally be avail- 
able. 

One of the problems with simply providing state estimates is that there may be too much information for the 
operator to cope with effectively. It is therefore important to examine what simplifications human operators 
tacitly use when they are presented state information, and which of these may be built into the decision mak- 
ing system to ease the job of the operator with negligible loss in system performance. This work concerns 
itself with building a simulation of a satellite recovery task, providing a decision aiding system based on pro- 
cess state estimation, and examining the simplification issues with respect to this task. 

Satellite Recovery Task 

A simulator is under construction which will include two satellites in orbit around the earth. One of 
these is a target of recovery, and it will be experiencing an arbitrary rotation initially without force or torque 
disturbances. The other satellite is a shuttle which the operator must position so as to fetch the target. The 
view’ from the shuttle of the target, stars, and the earth will be displayed to the operator, as will other infor- 
mation which might normally be available about the state of the shuttle and the target. The goal of the task 
is to slow or stop the rotation of the target, or to successfully plan actions so it may be grabbed with a robotic 
arm. 
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Although the mathematical description of rigid body rotation in three dimensions is straightforward and 
entirely deterministic, the observed behavior can appear to be quite complex and unpredictable. The decision 
aiding system will therefore incorporate a model of the rotating target, and. the human will provide data to 
the model as follows. A video frame may he taken at any time, and the operator will input the screen loca- 
tions of certain agreed upon reference points. After several frames have been processed in this way, the com- 
puter will have a refined estimate of the process state. A sufficiently refined state estimate can be used to 
estimates future states of the target for planning. 

Some Numerical Considerations 

We assume that the process state can be specified as a set of unknown values 0, through 0 n . The struc- 
ture of the process is well enough specified that any sot of values {0 lf • ,0„} for <“<o uniquely determines 
the {0,. .0, } for t >/„. This is equivalent to saying that there is some function <P: 0{t)—+0(t+dl). 

The function 0 describes the dynamic behavior of the process of interest. For the satellite example, the states 
of th<* process are the linear and angular positions and velocities of the satellite, and possibly some geometric 
or inertial parameters if they are not precisely known ahead of time. The <f> .'unction is a model of the process 
dynamics, so it would he normal to rhoo.se <0 to correspond to the equations of motion for a rigid body. 

In addition to the function <P which represents the process dynamics, there must be an additional function 
'f / ‘. 0(0 — *•'.( / ) which relates the states of the plant to observables. Again for the satellite example, the observ- 
ables are the (x.y) coordinate pairs of reference points as they appear on the video screen, and the mapping ^ 
expresses the relationship between the satellite’s position in space and the location of the satellite as it appears 
on the monitor screen. 

Considering only the problem of state estimation, there are approximately twelve state variables which will 
fully desrihe the state of the target, and twelve more to describe the state of the shuttle. Assuming that shut- 
tle state information is available to a high degree of accuracy, we can consider these latter variables as 
known and the problem reduces to that of estimating the values of twelve unknown target state variables. For 
each video frame which is taken and each reference point recorded, two values are provided, so a minimum of 
six reference point locations is needed to establish the state. Since the velocity information cannot be derived 
from a single frame, at least two frames of data must be taken. Additionally, the measurement process itself 
will contain noise, so more points than the absolute minimum should he used to achieve a better estimate. 

Algorithms such as the extended Kalman filter algorithm or stochastic approximation methods may he used to 
perform the state estimates. A significant feature of these methods is that error estimates are provided in 
addition to the point estimate of state. It is a major purpose of this work to determine of what use the addi- 
tional information may be to a human operator, or if it is of any use whatsoever. 

Progress to Date 

Previous work has indicated that when humans are presented with state estimates for a specially con- 
structed •■xperimenf al task, they make simplifications to it before using it in their decision process. “Parti- 
tioning" when a distribution over a large number of possible values of a state variable is reduced tonne 

over fewer possible values. For example, the reduced set of possible values could be “hot," "medium." and 
"■old." "I* int estimates" of states are single values derived from a distribution over possible values, "Focus" 
r fers to ■■Mijcentrating on the values of a single state variable while ignoring the information relating to 
another -tale variable. Fxperimental evidence has been gathered to indicate that people will use each of these 
■'implifivatious. and their role in the satellite retrieval task must be examined. 

A -imul-it. >r has been constructed which includes a shuttle and a target spacecraft. A numerically stable 
Emulation is achieved by separating the motion of a nominal, orbiting reference frame from the motion of the 
bodies within tin- reference frame. The simulation of the dynamics of rigid bodies in the non-inertial. orbiting 
reference frame must, of course, include effects due to the rotating reference frame, and the non-uniform grav- 
ity field. 

The simulation of rigid body rotation is usually accomplished bv using cider angles to represent tin* angular 
position of tit** body. This is a straightforward and practical method for simulations of, say. robotic arms, 
where good Emulations are required only for short periods of time and angular excursions are limited, hi the 
case of a tumbling body, the simulation must he accurate and stable over long periods, and arbitrary rota- 
tions must lie allowed. These two goals are difficult to achieve using a standard eider angle approach. This 
probh m Is solved using the angular momentum vector and a rotation matrix directly as state variables. 
I sing this method, simple second order eider integration yields very satisfactory results over a long simulation 
period. 
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A third Interesting area Is the direct recovery of velocity information from successive frames through image 
processing techniques. In this approach, the operator would indicate reference point locations as before, and 
he would also indicate a region in the image where velocity processing is to take place. Results in this area 
are preliminary. 

Conclusions 

A simulator is tinder construction which allows subjects to "fly" a spacecraft to a tumbling target satel- 
lite and retrieve it. IJecause of the nature of the information involved, a decision aiding system will almost 
certainly be useful in such a task. Some details relating to the simulator and state estimator have been 
worked out, and a few more remain before the research will be complete. It remains to be determined what 
simplifications of the state information are useful and appropriate to apply in this task. The results of previ- 
ous experimentation regarding simplification of state information by humans will be applied as it is possible 
to do so. 

6. A COMMAND LANGUAGE FOR TELEMANIPULATION 
Wael Yared 


General Approach 

To qualify as an intelligent autonomous system, a command language for manipulators in a supervisory 
control situation must include two distinct components: knowledge of the task at hand, and a systematic 
scheme of man- machine interaction . 

I am approaching the problem of designing a supervisory command language from a computational- 
linguistic point of view. In a first phase, a conventional, domain-specific natural language interface is being 
developed and implemented on a computer simulation. The simulation brings into play a graphic representa- 
tion of a manipulator arm in a specific task environment with obstacles, and an operator. The operator types 
task-level descriptions to the robot both of the environment and of the task itself. In a realistic situation an 
accurate description of the environment could be provided by CAD-!ike databases, and a rough outline of the 
obstacles by purposely vague linguistic descriptions quantified graphically using a frizzy set calculus. Task 
planning is then carried out interactively, in a mixed-initiative dialogue with the operator playing a super- 
visory role. This constitutes the second phase of the research, where the emphasis will be placed on develop- 
ing different grammars of discourse corresponding to the different structures of task-instruction dialogues. 

The Natural Language Interface 

A natural language interface is a necessity if one wants to exploit the vagueness inherent to human 
language. The scheme that has been adopted here is a simplified, context-sensitive grammar of English imple- 
mented using Augmented Transition Networks. The arcs in these networks embody syntactic features (such 
as noun phrases, prepositional phrases, etc.) as well as features more commonly identified with semantic 
grammars (which arc. in this implementation, features such as physical attributes of objects and obstacles in 
the environment). A new type of arc has been defined for "hedges", or linguistic modifiers, which operate on 
physical object attributes and are quantified by fuzzy set membership functions. Extra-linguistic information 
such as pointing is also supported in this application. No pragmatics is incorporated at this level, as this 
knowledge is relegated to the discourse grammar. 

The Discourse Grammar 

Research in discourse structure is a conceptual outgrowth of sentential linguistics. Researchers in the 
field have first attempted to find internal representations of utterances that are amenable to computational 
manipulation: this internal representation is usually the result of mapping from a parsed representation of a 
sentence to an expression in 'logical form". The logical extension of that work is the linking of such discourse 
entities, in other words developing some bookkeeping system (in the form of a pushdown stack for example) 
that stores and updates the successive "foci" of discourse, with an ability to point back to previous utterances. 

A practical discourse grammar can then be specified in the form of an Augmented Transition Network 
just like a regular grammar with the basic unit of analysis being, instead of the word, an utterance or a set of 
utterances. States and arcs in this network are written in terms of functional discourse relations (for example 
setting up a sub-topic of discourse, resuming a higher conversational level). From each state, the set of arcs 
leaving that state represents the conversational "moves" available to the conversational participant. The tests 
and registers, just like in a sentence ATN, are used to track the dynamic aspects of the discourse -the stack of 
foci. Finally, the actions on arcs serve to update the current focus of conversation. 
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This is a first attempt at systemizing man-machine interaction. It benefits from the linguistic planning 
ability of a skilled human operator to build up a task plan for the robot, and is particularly well-adapted to 
’’structured" task", such as the assembly or dissembly of a piece of equipment (a pump motor, say), but could 
conceivably be extended to less obvious situations as well. Experiments will be performed on transcripts of 
expert/apprentice instructional dialogues for .arious tasks in view of eliciting an underlying structure. 


7. DEVELOPMENT OF HUMAN-FLYABLE, REAL-TIME DYNAMIC COMPUTER- 
GRAPHIC SIMULATOR 

Chi-Cheng Cheng, Kan Chin, and Patrick Judd 

!n this proje t we are providing, on a Silicon Graphics IRIS workstation, a vehicle, manipulator arm 
and environment, all realistically desplayed in real time with hidden surface re .urn I. The vehicle has realis- 
tic dynamics a' d is controlled by a six-axis force-joystick. The manipulator arm has only kinematics 
represented; it is controllable either by a second six-axis force-joystick or by a six DOF master arm. The 
manipulator and end effector will be capable of grasping simple objects. A vehicle control panel is also 
displayed on the computer screen. 

One version of this simulation has a capability lor running repeated "test trials" with different control move- 
ments, recording the trajectory as well as the control inputs for each trial, and then having the "best" com- 
mand trajectory fed to the "actual" teleoperator to reproduce in reality the trajectory selected from the simu- 
lation. This technique, it seems to us, offers many possibilities as a simple adjunct to a telerobot. 
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